1. Field of the Invention
Methods and apparatuses consistent with the present invention relate to error concealment, and more particularly, to error concealment that can improve the picture quality of video data by concealing frame loss and reducing error propagation.
2. Description of the Related Art
In line with increasing transmission of moving images over the radio network environments, developments in the radio network technologies have also been making rapid progress. Currently, extensive research and developments have been made in technologies for compressing such moving image frames. Frame compression technologies, such as H.263, H.263+, MPEG-1, MPEG-2, MPEG-4 have been established as international standards by the International Standards Organization. Since data produced using these compression technologies has high compression rates, it can be easily transmitted even in systems that are provided with channels having limited bandwidths.
However, in the case where the data compression rate is high, data being restored may be severely damaged due to an occurrence of error in the transmitted data. That is, packet loss frequently occurs due to a radio-wave interference caused by obstacles, and this causes a loss of video data to occur. In order to heighten the compression efficiency of the video data being transmitted, data of the previous frames are referred to. Accordingly, after the error occurrence due to the loss of video data, an error propagation phenomenon continues. In particular, the frame loss causes a serious error and error propagation phenomena in comparison to the slice or macroblock errors.
In order to overcome the above-described drawbacks, many attempts to encode moving image (hereinafter referred to as “video”) data so that it has error resilience have been made in many fields. A frame error concealment technique has been proposed as one method for realizing this resilience. The frame error concealment is a method for reducing the influence of an error when video frames, in which the error exists, are received and decoded by a video decoder. This error concealment is different from error correction, which is a method of correcting incorrect data.
Among error concealment techniques, a temporal prediction error concealment conceals an error by using redundant information between successive frames according to video information, and spatial prediction error concealment conceals an error in the same frame, including lost information, by using spatial redundancy. Spatial prediction error concealment compensates for damaged macroblocks by using information of undamaged macroblocks in the present frame without using the temporal redundancy. For this, error concealment techniques which replace the lost block by a neighboring macroblock or which uses various interpolation methods have been proposed. Temporal prediction error concealment is a technique for referring to a frame existing in a temporarily different place in order to restore information of the lost macroblock in the present frame by using the temporal correlation between successive frames. Temporal prediction error concealment can be classified as a single-frame error concealment where one frame is referred to during the restoration of the video frame, and a multi-frame error concealment where two or more frames are referred to.
Several frame error concealment techniques have been proposed according to a method for obtaining a motion vector of a lost macroblock. One method uses a previously received frame, and another method uses two or more previously received frames. The method that uses a previously received frame may be further classified into three methods according to the method of using the previously received frame (hereinafter referred to as a “previous frame”) as follows.
The first method uses a macroblock in the same position in a previous frame. The second method restores the lost motion vector by using an intermediate value or an average value of motion vectors neighboring the lost macroblock of the presently received frame. The third method uses a macroblock that is most similar to the present frame by estimating the neighboring pixel values of the lost macroblock from the previous frame.
The multi-frame error concealment technique restores the motion vector using the methods used for the single frame concealment technique. However, in order to find the lost macroblock, the multi-frame error concealment technique does not search for only one previous frame, but for at least two previous frames.
Prior to the explanation of the problems of a related art frame error concealment technology, the construction of a general video decoder will be briefly explained in the following. Generally, a video decoder restores the original video signal by performing a variable-length decoding and an inverse quantization of a compressed video bitstream. FIG. 1 is a block diagram illustrating the construction of a related art video decoder.
Referring to FIG. 1, a buffer stores an input bitstream, and a variable length decoding unit 12 performs a variable-length decoding of the bitstream stored in the buffer, and outputs video information generated as a result of the variable length decoding to an inverse quantization unit 13 and a motion compensation unit 15. The inverse quantization unit 13 receives quantization information output from the variable length decoding unit 12, and performs an inverse quantization of the variable-length-decoded data. An inverse discrete cosine transform (IDCT) unit 14 performs an IDCT of the inverse-quantized data output from the inverse quantization unit 13. A motion compensation unit 15 generates motion-compensated video data using the motion information and the video data output from the variable length decoding unit 12. An addition unit adds the video data motion-compensated by the motion compensation unit 15 and the video data inverse-DCT-transformed by the IDCT unit 14. A storage unit 17 stores the video data output from the addition unit 16.
As described above, the video decoder as illustrated in FIG. 1 performs the variable length decoding by reading the video data according to the decoding capability of the video decoder after storing the input bitstream in the buffer 11. However, if an error occurs during the transmission of the video data and lost data, which cannot be variable-length-decoded, is included in the input video data, the lost error data accumulates to affect subsequent images.
FIG. 2a is a view explaining the error propagation when a frame loss occurs in the related art video decoder. If an entire frame is lost, as depicted by numeral 21 in FIG. 2a, this affects macroblocks that refer to a forward frame causing serious picture quality deterioration. That is, the lost frame affects all the following frames 22, 23 and 24 causing the picture quality to deteriorate. In order to overcome the drawback due to the error occurrence as described above, a frame error concealment method has been proposed as one method for realizing error resilience. In the whole description of the present invention, the frame error concealment means a method for reducing the influence of an error in decoding subsequent data if the error occurs and an entire frame is not received in the decoder.
FIG. 2b is a view explaining the method of referring to macroblocks of the previous frame in a related art frame-unit error concealment method. This method brings a macroblock 30-1 of the previous frame 30 that corresponds to a macroblock 40-1 of a lost frame 40 by applying a zero-point motion vector 35 to the previous frame 30 in order to restore the macroblock 40-1 of the lost frame 40.
However, the method as illustrated in FIG. 2b refers to the data of the previous frame 30, and thus if a scene is changed or there is a great change between frames, a large prediction error occurs reducing the picture quality.
FIG. 2c is a view explaining a method of predicting pixels of a lost frame through bidirectional extrapolation in a related art frame-unit error concealment method.
The method of FIG. 2c performs a bidirectional concealment reference, i.e., a forward concealment reference and a backward concealment reference. The forward concealment reference operates as follows.
The forward concealment reference performs an extrapolation of motion vectors in the unit of a macroblock with respect to the previous frame so that the previous frame corresponds to the lost frame. In FIG. 2c, the reference numeral 55 is a motion vector of a macroblock 60-1 of a frame 60. Extrapolation is performed with respect to a frame 70 after a motion vector 55, and a motion vector 65 that corresponds to a pixel 70-0-1 is obtained. Accordingly, a pixel 60-0-1 in the frame 60 is selected as a concealment reference point of the pixel 70-0-1 in the frame 70. Through this process, the pixel 70-0-1 may have more than zero concealment reference points with respect to the previous frame 60. If no concealment reference point exists, the pixel in the same position in the previous frame 60 is determined as the concealment reference point. By contrast, if plural concealment reference points exist, an average point of the concealment reference points is determined as the forward concealment reference point.
The backward concealment reference operates as follows.
Concealment reference points 80-0-1, 80-0-2 and 80-0-3, which correspond to the pixel 70-0-1 of the lost frame, are searched for using motion vectors 75, 76 and 77 of respective macroblocks of a backward frame 80 as shown in FIG. 2c. The following process is the same as the forward concealment reference described above.
The average value of the forward and backward concealment values obtained through the above process becomes the concealment value of the pixel 70-0-1 of the lost frame 70. The lost frame 70 can be restored by performing the bidirectional reference for the respective pixels.
However, the method as illustrated in FIG. 2c has the following problems.
First, its prediction accuracy is degraded due to the bidirectional reference during a scene change. If the scene is changed in the backward frame of the lost frame, there would be no correlation between the lost frame and the backward frame, while if the scene is changed in the forward frame of the lost frame, there would be no correlation between the lost frame and the forward frame. Since the method as illustrated in FIG. 2c performs the bidirectional reference in the cases as described above, the prediction accuracy for the respective pixels of the lost frame is lowered.
Second, since the error is concealed using only one forward frame when the forward reference is performed, the probability of predicting accurate pixels is lowered when the concealment of an edge part of the image is performed or when an object in the image disappears and then appears again.
Third, since the prediction is performed by simply obtaining the average value with respect to the reference points that conceal the lost frame, inaccurate information of the reference points have the same weight value in performing the prediction which lowers the prediction accuracy.